Managing Mutual Information & Transfer Entropy In Synthetic Ecologies

In this paper we consider transfer entropy and mutual information in terms of their application in the emerging highly interconnected and dynamic synthetic ecologies underpinned by the Cyber. We consider existing models relating to the management of learning and change within organizations and as they may relate to mutual information (MI) and transfer entropy (TE) within socio and info/techno settings, based upon a Mech-Organic perspective. A premise of this paper is that change is costly and that it needs to be seen through a social as well as an info/techno lens. We identify potential improvements to existing models and applications applied to the management of change by considering alternative models and how they may be applied collaboratively within a learning organization

Transfer Entropy as a Log-likelihood Ratio

Transfer entropy, an information-theoretic measure of time-directed information transfer between joint processes, has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields, including the neurosciences, ecology, climatology and econometrics. We show that for a broad class of predictive models, the log-likelihood ratio test statistic for the null hypothesis of zero transfer entropy is a consistent estimator for the transfer entropy itself. For finite Markov chains, furthermore, no explicit model is required. In the general case, an asymptotic chi-squared distribution is established for the transfer entropy estimator. The result generalises the equivalence in the Gaussian case of transfer entropy and Granger causality, a statistical notion of causal influence based on prediction via vector autoregression, and establishes a fundamental connection between directed information transfer and causality in the Wiener-Granger sense

Information flow in a kinetic ising model peaks in the disordered phase

There is growing evidence that for a range of dynamical systems featuring complex interactions between large ensembles of interacting elements, mutual information peaks at order-disorder phase transitions. We conjecture that, by contrast, information flow in such systems will generally peak strictly on the disordered side of a phase transition. This conjecture is verified for a ferromagnetic 2D lattice Ising model with Glauber dynamics and a transfer entropy-based measure of systemwide information flow. Implications of the conjecture are considered, in particular, that for a complex dynamical system in the process of transitioning from disordered to ordered dynamics (a mechanism implicated, for example, in financial market crashes and the onset of some types of epileptic seizures); information dynamics may be able to predict an imminent transition

COMPLEXITY: METRICS AND MODULES

Complex systems are of vast importance in the practical world as well as presenting many theoretical challenges. The measurement of system complexity is still imprecise. For many systems, their modular construction brings challenges in understanding how modules form and the emergent behavior which may result. In other systems, it is the development of encodings and communication protocols which allow complexity to increase dramatically. We take a broad view of these issues and then consider the nature of the system space which generates complexity. We show examples from cellular automata and applications of neural networks to data mining which suggest that complex systems often occupy simple structured sub-spaces. Finally, we look at the way modularity relates to networks and the implications for understanding human cognitive processing.Agents, cellular automata, complexity, modularity, Go, neural networks, density problem